Telepresence conference room layout, dynamic scenario manager, diagnostics and control system and method

ABSTRACT

A method of conducting an audio-visual conference among a number of conference rooms comprising storing predetermined conference configuration data in a computer system for a plurality of room types and a plurality of combinations of configuration factors, receiving conference variable data, generating a room list of included conference rooms, generating interconnection data for each included conference room according selected by said a computer system performing a recursive process until a feasible set of configuration data is found which permits interconnection to every other included conference room, and if no applicable set of configuration data is feasible, the distribution process returns to a next higher room in said room list.

FIELD OF THE INVENTION

This invention pertains to the field of teleconferencing andtelepresence systems and methods and, in particular, to dynamicteleconferencing and telepresence systems and methods encompassingmultiple locations.

BACKGROUND AND SUMMARY OF THE INVENTION

Audio/video teleconferencing has become an important and effectivemethod of communication between individuals and groups over greatdistances. While prior (“legacy”) audio/video teleconferencing systemshave provided improvements over audio-only conferencing methods (forexample, telephone conferencing), the difference between in-personconferences and the legacy audio/video teleconference systems remainssubstantial.

It is known that a significant amount of communication betweenindividuals occurs in non-verbal form (such as eye contact, facialexpressions, “body language”, hand gestures, and other non-verbal cues).It has been found that the legacy teleconferencing systems do noteffectively convey such non-verbal communication. Thus, there is a needfor an improved teleconferencing system and method that more closelyimitates an in-person conference.

The telepresence system and method of the present invention fulfill theneed for an improved audio/video teleconferencing system by providingtelepresence conference rooms which closely mimic an in-personconference, and by providing a system and method for automaticallyoperating conferences held between such telepresence conference rooms,including the scheduling, configuration, initiation, diagnosis,monitoring, fault correction, and termination of such conferences. Thetelepresence system and method may also incorporate legacyteleconference rooms into a telepresence conference.

BRIEF DESCRIPTION OF THE DRAWINGS

For a complete understanding of the above and other features of theinvention, reference shall be made to the following detailed descriptionof the preferred embodiments of the invention and to the accompanyingdrawings, wherein:

FIGS. 1-3 are plan views of telepresence conference rooms constructed inaccordance with the invention;

FIG. 4 is a schematic representation of a configuration table employedby the system;

FIG. 5 is a schematic representation of a 3-way teleconference;

FIG. 6 is a schematic representation of a 4-way teleconference;

FIG. 7 is a schematic representation of a 5-way teleconference;

FIG. 8 is a schematic representation interconnections between equipmentin the rooms in a 3-way teleconference;

FIG. 9 is a schematic of a telepresence conference network; and

FIG. 10 is schematic of a telepresence site constructed in accordancewith the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention is related to the teleconference system andmethods disclosed in the applicant's PCT patent application Ser. No.PCT/US08/54013 and applicant's U.S. provisional patent application Ser.No. 60/889,807, the disclosures of which are hereby incorporated intheir entirety by reference.

Telepresence Conference Room Layout

Referring to FIGS. 1 and 2, telepresence conference rooms of the presentinvention are carefully designed to closely mimic an in-personconference held at a conference table. To closely mimic an in-personconference, most of the attributes of the telepresence room arecarefully predetennined and controlled, including the general roomlayout, the table size, shape and position, the maximum number ofparticipants and their seating positions, the number, size and locationof participant displays, the location of any alternative display(s)(e.g., laptop computer presentation screens), the location of anylecterns, the lighting controls, the microphone and speaker placement,the room acoustics, the operation of windows and shades, the cabling,the climate, the color, pattern and texture of walls, the furniture andother room surfaces, and the overall aesthetics of the room.

The room layout is affected by a number of factors, including the numberof participants in a room, the size of the conference table, the angleof the view of the cameras, the amount and type of the video displays,whether the displays are mounted on a wall or on a free-standingsupport, and other factors.

Participant Displays

Each telepresence conference room of the present invention has aplurality of large, flat-screen participant displays 16, 17, 18, 19mounted to a wall or to a free-standing support structure. Preferably,each participant display is a high-definition, wide-format (16:9 aspectratio), LCD or plasma display and is substantially identical in size andappearance to the other participant displays in the room. Preferably theparticipant displays have a diagonal active display dimension of greaterthan about 46″.

The participant displays 16, 17, 18, 19 in a given telepresence room arepreferably mounted at a consistent, predetermined height off of thefloor such that the bottom of the active display area of eachparticipant display is about equal in height to the top of theconference table, or, between about 27 inches (68.5 cm) and about 30inches (76.2 cm) off of the floor.

Preferably, there are at least four such participant displays in eachtelepresence room, with two “central” participant displays 17, 18arranged symmetrically on either side of a longitudinal (front-to-back)axis of symmetry 100 of the telepresence room, and further participantdisplays located on either sides of the central participant displays.Alternatively, there may be 1, 2, 3 or 5, or some other number ofparticipant displays. The participant displays are arranged such thatadjacent displays closely abut one another, thereby forming ahorizontally-elongated, contiguous or nearly-contiguous compositeparticipant display. Each participant display is preferably assigned aposition indicator in the system, for example 0, 1, 2 and 3, from leftto right, for a 4-display room.

Preferably, the distance between active display portions of adjacentdisplays is less than or equal to about one inch to approximate theappearance of a contiguous or near-contiguous composite participantdisplay. Aesthetic border portions of the individual participantdisplays (i.e., inactive areas or “bezels”) located between activeportions of adjacent individual displays are removed or reduced tominimize the distance between the active portions of the adjacentdisplays. Each participant display is fixed at a predeterminedorientation about a vertical axis such that the plurality of displaysforms a gentle arc which is concave (open) toward the conference table102, such that the arc is similar in shape to an adjacent edge 105 ofthe conference table and such that the arc has a center of curvature onthe longitudinal axis 100 of the telepresence room 102, which center ofcurvature is located behind the seating positions of the conferenceparticipants.

Participant Cameras

Referring to FIG. 3, each telepresence conference room preferably hasone or more participant cameras 26, 27, 28, 29 for each participantdisplay 16, 17, 18, 19 located on or close to the center of the top edgeof an associated display. Certain rooms may have additional cameras, forexample eight cameras and four displays, for use in particular modes ofoperation, such as the ‘executive’ mode described below. A room suitablefor operation in ‘Executive’ mode may have four additional cameras 27′,27″, 28′, 28″ located generally on the right and left sides of the topedges of the center displays 17, 18. In an ‘Executive’ room, theadditional cameras 27′, 27″, 28′, 28″ may share equipment, such as codecequipment, with other cameras in the room. For example camera 26 and 27′may be connected to the same codec, since only one of the two cameraswill be operational at any given time. Similarly, other camera pairs mayalso share codec equipment, such as cameras 27 and 27″, 28′ and 28, and28″ and 29.

Each participant camera is preferably assigned a position indicator inthe system, for example 0, 1, 2 or 3, from left to right, for a 4-cameraroom, or 0, 1, 2, 3, 4, 5, 6 or 7, for an 8-camera room.

Each participant camera may be movably mounted to a linear or arcuatetrack that permits manual or motorized movement of the camera relativeto a lateral (side-to-side) axis of the displays, or each camera may befixed relative to the lateral axis of the displays. Further, each cameramay include manual or motorized pan and tilt, to alter the optical axesof the camera, and manual or motorized zoom, to alter the optical fieldof vision of the camera (pan, tilt and zoom may be collectively referredto as PTZ).

Alternatively or additionally, each participant camera may be alarge-format, high-resolution digital camera which is located in a fixedposition relative to the lateral axis of the displays and which hasfixed optical axes and a fixed (wide) optical field of vision. Theimages captured by such digital cameras may be digitally cropped asneeded to select and optimize the images transmitted to and displayed atthe remote conference rooms. Further, the optical field of vision ofcertain participant cameras may overlap, which provides redundancy forfault tolerance and which provides flexibility in the selection ofcamera angles.

The optical field of vision of each participant camera is preferably atleast about 85 degrees to about 90 degrees in a horizontal direction andat least about 67 degrees in a vertical direction. Preferably there is aparticipant camera located on or nearly on the longitudinal axis 100 ofthe telepresence room (i.e., at the center-most position of thecomposite participant display), which camera has optical field of visionsufficient to capture an image of all participants seated at theconference table.

Conference Table & Seats

Preferably, the conference table 102 is substantially symmetrical abouta longitudinal (front-to-back) axis 104 of the table, which lies on, orsubstantially on, the longitudinal axis 100 of the telepresence room.The facing edge of the table (which faces the participants) is curved orarcuate as viewed from above such that it is convex toward theparticipants. Specifically, the facing edge 106 has a center ofcurvature which lies on the longitudinal axis 104 of the table (and thetelepresence room) and which lies substantially behind the participantdisplays.

The conference table 102 has a plurality of predetermined seatinglocations 108 (and associated chairs) located at predetermined locationsdisposed along the facing edge of the conference table. Preferably, eachseating location 108 is delineated by visibly distinctive and/or tactilemarkings 110 on or adjacent the facing edge of the conference table 102.As depicted, there is preferably at least one such marking 110 betweeneach seating position and at least one to the outside of the extremeleft and right seating positions.

Alternatively or additionally, the markings 110 can include or consistsof seating position lights, such as LEDs or the like, which areautomatically illuminated to indicate the proper seating position forparticipants, as determined by the system. For example, if only twoparticipants are present in a conference room having a table with eightseating locations, the system preferably illuminates the seatingposition lights associated with (i.e., between and on either side of)the two central seating locations located on either side of thelongitudinal axis 104. Likewise, the system preferably indicates thedesired seating location for all participants in a room.

The seating position lights are preferably connected to or controlled byseating position light control equipment located in the telepresenceroom or the site thereof. The seating position control equipment isactivated by the control equipment (and computers) of the systemoperator during initiation of a telepresence conference. In particular,the configuration computers of the system operator determines theoptimal and/or desirable seating positions for conference participantsaccording to various criteria (such as the number of and configurationof the other rooms in the conference) and illuminates the appropriateseating position lights in each room at the start of the conference toguide the participants to the correct seating positions.

Conferences conducted using the system often relate to sensitive topicsthat must remain confidential. The markings 110 may also consist of orinclude privacy indictors such as lights or LEDs or the like, which arecapable of displaying at least two distinct colors and/or appearances,such as Red and Green to indicate the current privacy status of theconference. The alternate color/appearance of the privacy indicatormarkings 110 is intended to indicate to the teleconference participantswhether the conference is in private mode where the audio and videosignals are excluded from monitoring by any central conference ornetwork management personnel, such as at a Network Operations Center(NOC). In particular, the privacy indicators may glow red during initialset-up of the conference (or when the assistance of conferencemanagement personnel is requested to modify the conference or solve atechnical problem) to indicate that the conference is in public mode.However, if conference privacy has been requested or selected by one ofthe conference participants, when the conference has been successfullyinitiated and conference management personnel no longer have access tothe video and audio streams, the privacy indicators may glow green toindicate that the conference is in private mode. As can be appreciated,the privacy indicators provide a convenient and readily discernibleindicator to all participants as to the current privacy status of thecall. Preferably, the privacy indicators in a particular room (or allrooms) are synchronized such that they all indicate the same status atany given time.

The privacy indicators are preferably connected to privacy indicatoractuation equipment at the site of each telepresence room, such as aspecial purpose computer (or general purpose computer with specialprogramming) suitable to actuate (e.g., change the color and/orappearance) of the privacy indicators in response to a command. Theprivacy indicator actuation equipment is connected to or accessible bythe Network Operation Center. Preferably, the system includes a privacycontrol which selectively prevents personnel of the Network OperationCenter from accessing audio and video streams when users of the systemdesire or select the conference to be in private mode, and whichselectively permit the NOC personnel to access the streams when users ofthe system do not desire privacy (i.e., when the conference is in publicmode). The privacy control may be located in the Network OperationCenter. Additionally or alternatively, the privacy control may bepresent in (or only in) one or more of the rooms in the conference suchthat the users of the conference have direct and convenient (andoptionally exclusive) access to the privacy controls. Preferably, thesystem automatically directs the privacy indicator actuation in eachroom in the conference to indicate the current privacy status of thecall, in response to receipt of a command via the privacy control. Wherepart of the privacy control is located in a room or site participatingin the conference and the conference is in privacy mode, the systempreferably monitors or polls such room-based privacy control on aregular or periodic basis to determine whether the privacy controlresponds with an indication that the conference is (still) in privacymode. If the privacy control responds that the conference is no longerin privacy mode (i.e., has been switched to public mode), or if theprivacy control fails to respond within a predetermined period of time(e.g., 1-60 seconds), the system preferably switches the conference outof private mode and into public mode.

The privacy control preferably includes a special purpose computer (or ageneral purpose computer with special programming) suitable to receivean instruction from a human user or operator to activate or deactivateprivacy, to selectively isolate or connect NOC personnel to the audioand video streams of a conference, and to issue appropriate commands tothe privacy indicator actuation equipment located at the sites of therooms in the conference.

Preferably, there are an even number of participant seats 108 at theconference table (e.g., 4, 6, 8, 10, etc.), and the seats are arrangedsymmetrically on either side of the longitudinal axis of the table. Eachparticipant seat is preferably allocated a predetermined andsubstantially equivalent amount of space (about 30″ or about 762 mm),with an additional separation buffer between the two center-mostparticipant seats located on either side of the longitudinal axis of thetable (about 6″ or about 153 cm). Thus, the minimum lateral dimension(length) of the table is dictated by the number of participant seats atthe table. For example, a 4-participant telepresence conference tablewould have a lateral dimension of greater than or equal to about 126″(4×30″+6″, or about 3200 mm).

The table 102 is positioned relative to the participant displays 16, 17,18, 19 such that a facing edge distance (as defined by the distancebetween the facing edge of the table and the center-most position of theparticipant displays, measured along the longitudinal axis of thetelepresence room) is sufficient to permit a single participant cameralocated at the center-most position of the composite participant displayand having a horizontal optical field of vision of about 85 degrees toabout 90 degrees to capture a complete image of the conference table,including a maximum number of participants seated at the conferencetable. Preferably, the facing edge distance is not substantially greaterthan the minimum distance necessary and is at least within 10 or 20percent of the minimum distance. Thus, for example, for conferencetables having maximum seated participants of 4, 6, 8 and 10 persons, theminimum facing edge distance would preferably be about 8 ft (2.5 m),about 10 ft (3 m), about 12.5 ft (3.8 m), and about 17 ft (5.2 m),respectively.

It is preferably that the conference table 102 (and especially the topsurface 112 of the table) have a substantially solid, light color (suchas maple or another light wood color), without substantial multiplecoloring or substantial visible wood grains, and should minimize lightand sound reflections.

For telepresence rooms accommodating 12 or more participants (e.g., 12,16, 18, 20 participants, etc.), the seating is preferably divided inhalf with the first “row” of seating being located at the first (orprimary) conference table 102 and a second row being located in a secondtier of seating at a second conference table 116. As with the firstconference table, the second conference table has predetermined seatinglocations and seating markings, and has an arcuate facing edge 106 whichis convex (as viewed from above) toward the participants in the secondtier. The opposite edge 118 of the second conference table (facing theparticipants seated at the first conference table) is preferably concavetoward the facing edge of the first conference table such that there isa substantially consistent distance between the opposite edge of thesecond conference table and the facing edge of the first conferencetable. Preferably, such distance is between about 1 m and about 2 m suchthat participants seated at the first conference table have sufficientroom to access their seats and to maneuver their chairs as desired.

Alternative Presentation Displays

The telepresence room may include an alternative presentation display120, such as a display for a laptop computer or DVD presentation. Suchalternative presentation display 120 is preferably mounted to one of thewalls 122 adjacent one of the lateral sides of the conference table 102,or to a support adjacent one of the lateral sides of the conferencetable. Preferably, the bottom of the alternative presentation display issufficiently high to permit participants seated at the far end of theconference table to view the entire active portion of the alternativepresentation display, anticipating that the conference table will oftenbe populated with other participant's laptops. It has been found that asuitable distance from the floor for the active display portion ofalternative presentation displays is about 40″ (1 m) off of the floor.

Audio & Acoustics

Preferably, a plurality of cardioid, mini-boundary participantmicrophones are permanently installed in the top surface of theconference table to capture conversations between the conferenceparticipants. The microphones preferably have a small “footprint” ofabout 1″ (radius) and project upward from the top surface of the table ashort distance of less than about one-half inch (½″), such that themicrophones are inconspicuous. The microphones are located on or near alateral (side-to-side) center axis of the conference table atsubstantially equal intervals along the lateral axis such that thedistance between any seated participant and the nearest microphone isbetween about 20″ (58.8 cm) and about 30″ (76.2 cm). Preferably, themicrophones have a sufficient sensitivity and fidelity to capturenormal-volume conversations within such range (i.e., about 60 dBsensitivity).

Further, each telepresence room includes a plurality of speakers,preferably located behind or adjacent each of the participant displays.Preferably there is a speaker for each individual participant display inthe composite participant display, and each speaker is located in aboveor below one of the individual participant displays (and preferably inclose proximity), such that there is a one-to-one association ofspeakers to individual participant displays. As will be discussed infurther detail below, the audio and video signals from each remotetelepresence room are reproduced/displayed at similar locations toenhance the approximation of an in-person conference. Preferably, eachparticipant speaker is assigned a position indicator in the system, forexample a number 0, 1, 2 or 3, from left to right facing the participantdisplays, which corresponds to, and is preferably the same as, theposition indicator of an associated participant display and camera pair.For example, participant speaker ‘0’ is associated with participantdisplay ‘0’ and participant camera ‘0’, and so on.

Preferably, the system includes audio processing electronics to minimizeor eliminate undesirable audio effects (such as echoes and feedback)produced in the room, which electronics may be located at eachtelepresence site and/or at a central location. To minimize suchundesirable audio effects, the acoustics of the room should be designedsuch that the room has a Noise Criteria of less than or equal to about35. To achieve the desired acoustics, at least one of any two parallelhard surfaces (such as the conference table top and the ceiling, orparallel walls) should be acoustically treated.

Lighting

Preferably, the lighting at each telepresence room is controlledremotely by the telepresence system, such as with wireless (e.g.,infra-red) controls, low voltage relay closures, Telnet sessions and/orRS232/422/485, or other suitable means. Preferably, the intensity of thelighting is evenly distributed throughout the portions of the room thatare to be captured on video by the cameras, including the participants,the table, the back wall and up to 50% of both side walls.

Teleconference Control

Referring to FIG. 9, a telepresence conference may include a pluralityof telepresence rooms/sites 10, 12, 14 and an LVC room 15′, connected toa bridge telepresence room, such as telepresence room 10.

The telepresence conferencing equipment at each telepresence site 10,12, 14 is preferably connected to a system operator 37 via a private orpublic network 38, such as the Internet. Preferably there existsalternate connections between the system operator 37 and eachtelepresence site 10, 12, 14, such as via dial-up modem connectionsand/or ordinary telephone connections (not shown). One of thetelepresence rooms 10 may be a bridge to an LVC room 15′, such that theother telepresence rooms 12, 14 in the conference connect to the LVCroom 15′ through the bridge telepresence room 10.

The system operator 37 preferably is preferably located at a (single)central location (site) and includes a Network Operations Center (NOC).The system operator includes several general purpose and special purposecomputers (running autonomously and/or at the direction and control ofhuman operators) which computers are operable to determine the proper oroptimal configuration for a teleconference and to remotely configure,interconnect and manage teleconference equipment, including thetemporary interconnection of teleconference equipment located in eachtelepresence room and/or at the site of the system operator.

Referring to FIG. 10, at the heart of each telepresence site is aremotely-configurable 16×16 Audio/Video matrix switch 39, which isconnected to the network 38 and to which most other devices at thetelepresence site are connected.

Each telepresence site also preferably includes:

a plurality (e.g., four) of each of participant microphones 40,participant speakers 41, audio/video codecs 46, participant cameras 42,and participant displays 44 (the participant cameras and displays beingin pairs), where each is connected to the 16×16 A/V matrix switch 39(either directly or indirectly through other equipment) and where theA/V codecs 46 are connected to the network 38;

each codec is preferably assigned a (logical) position indicator in thesystem, for example 0, 1, 2 and 3 (see FIG. 8), which corresponds to theposition indicator of the set of participant camera, participant displayand participant speaker to which the codec is connected;

if the participant displays and/or cameras are digital, they and thevideo ports of the codecs are connected to the 16×16 digital videomatrix switch;

an alternate presentation display 48 connected to the 16×16 A/V matrixswitch 39;

a mutli-channel audio processing unit 50 connected intermediate theparticipant microphones 40 and the 16×16 A/V matrix switch 39 andconnected to the A/V codecs 46 (where the audio signals of the A/Vcodecs 46 are sent to a reference bus of the audio processing unit suchthat they are used as echo canceling reference signals);

a multi-channel (e.g., 4-channel) audio amplifier 54 connectedintermediate the participant speakers 41 and the 16×16 A/V matrixswitch;

a plurality of (e.g., four) audio alarm relays 56 connected intermediatethe multi-channel audio amplifier 54 and the 16×16 A/V matrix switch 39;

an 8×1 diagnostic switch 58 having a plurality (e.g., four) incomingconnections from the 16×16 A/V matrix switch 39;

a diagnostic microphone 59 (or alternatively or additionally a signalfrom one of the participant microphones 40 prior to the audio processingunit 50) connected to an input of the 8×1 diagnostic switch 58;

an audio server 60 connected to an output of the 8×1 diagnostic switch58;

an audio alarm relay 62 connected intermediate the audio server 60 andthe 8×1 diagnostic switch 58;

a audio meter graphic generator 64 connected intermediate the audioserver 60 and the 8×1 diagnostic switch 58;

a diagnostic test tone generator 66 (e.g, an oscillator) connected tothe 16×16 A/V matrix switch 39;

a room view camera 68 connected to the 16×16 A/V matrix switch 39;

a single-source video server 70 connected to the room view camera 68 andto the 16×16 A/V matrix switch 39;

a multi-source video server 78 (e.g., four-source video server)connected to each of the audio/video codecs 46;

an LVC codec 80 connected to the 16×16 A/V matrix switch 39;

an audio program playback device 82 connected to 16×16 A/V matrix switch39; and

an integrated control system 84.

Telepresence Conference Dynamic Scenario Manager

The configuration of telepresence conferences can encompass a very largenumber of permutations depending on factors such as the number ofparticipating conference rooms, the number of participants in each roomand the number of individual participant displays in each room. Thetelepresence system employs a scenario algorithm to dynamicallydetermine the optimal configuration for the telepresence conference forany given set of variable parameters, and which automatically generatesand transmits configuration instructions to the various components ofthe telepresence system to set up and initiate the conference.

Certain characteristics are predetermined and known to the telepresencesystem, such as the identities of the telepresence rooms within a groupof telepresence rooms that may be interconnected via the system, thenumber of participant cameras and participant displays in each of thoserooms, and the presence or absence of any alternative presentationdisplays.

To configure a telepresence conference, a user (or an operator) enterscertain conference-variable data into a terminal connected to thetelepresence system, such as the name of the owner/customer of the groupof interconnectable telepresence rooms, a list the rooms to be connectedin the conference, the number of people in each room, the desired numberof video streams to use in the conference, the identity of the room fromwhich an alternate presentation will start (if any), the type of suchalternate presentation (e.g., VGA-laptop or DVD-video), the identity ofthe LVC bridge room (if a Legacy Video Conference—ISDN system is in theconference), the date & time to initiate the conference, and theduration or time to terminate the conference. Preferably, suchconference-valiable data may be entered via a computer connected to ahost computer system via a local area network (LAN) and/or a wide areanetwork (WAN), such as the Internet or a private network.

The scenario algorithm of the telepresence system eliminates theheretofore laborious and costly process of manually referencing andentering conference configuration connection into the telepresencesystem.

A particular benefit of the present system is the ability to handle awide variety of room configurations with equal or differing numbers ofcameras and displays. Further, the system is able to accommodatedifferent modes of operation for each room. The system employs aplurality of pre-defined room configurations for each room type, forseveral possible variations in telepresence call scenarios. New roomtypes and scenarios can be rapidly added to the system as desired byadding new pre-defined room configurations, without the need to re-writethe primary software for the system. In particular, new room types maybe added with D displays and C cameras (where D and C may or may not beequal) without changing any code, just by adding configurations for thenew room types. Further, new operation modes can be added withoutchanging code. And, the invention contemplates the use of a graphicaluser interface to facilitate the addition and modification ofconfigurations for room types.

System Initialization (Determination and Input of Optimal Behavior ofRooms)

Prior to configuration of a teleconference, certain configuration datais entered into the system to allow the system to determine optimalconfigurations for the various teleconference scenarios.

Configuration factors that influence proper vectoring in a telepresenceconference include: (a) the number of telepresence rooms in theconference, (b) for each telepresence room in the conference: (1) thenumber of active (i.e., working) participant displays, (2) the number ofactive participant cameras, and (3) the number of participants, (c) thenumber of LVC conferences in the conference (if any), (d) the maximumdesired streams between any two telepresence rooms in the conference, aseither selected by the user or the system, and (e) the operation mode,which can be any one of a number of pre-defined operation modes.

There is a room type defined for each unique combination of activeparticipant displays and active participant cameras. Room types arepreferably stored in a room_type table as set forth in Table A below. Ascan be appreciated, the system preferably has a plurality ofpre-determined room types (e.g., 7 room types). However, room types maybe added or removed from the system by adding or removing room typesfrom the room_type table.

TABLE A Room_Type_ID No. of Active Displays No. of Active Cameras 1 1 12 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 4 8

The operation modes are also pre-defined by the system and are stored ina operation_mode table as set forth in Table B below. As with roomtypes, the system preferably has a plurality of pre-determined operationmodes (e.g., 2). However operation modes may be added or removed fromthe system by adding or removing operation modes from the operation_modetable.

TABLE B Mode_ID Mode Description 1 Normal 2 Executive

Every room type has at least one configuration, which defines howconnections are made with the room and other rooms in the call. Theconfigurations are stored in one or more configurations tables as setforth in Tables C, D, E and F below.

FIG. 4 depicts a schematic representation of a record in a configurationtable. Each record in a configuration table includes fieldsrepresenting:

-   -   a room type value (r_type)    -   an operation mode value (op_mode)    -   a value indicating the total number of rooms in the call,        including any LCV conferences (way)    -   the maximum number of streams between any two telepresence rooms        in the call (stream)    -   the number of out-going streams to other telepresence rooms in        the call, from left to right (streams_comb_out)    -   the out-going stream ID numbers assigned to the other rooms,        from left to right, where “0” is the left-most stream        (streams_out)    -   the number of incoming streams from other telepresence rooms in        the call, from left to right (streams_comb_in)    -   the incoming stream ID numbers assigned to the other rooms, from        left to right (streams_in)    -   the priority of the configuration with respect to other        pre-defined configurations for the same combination of room        type+operation mode+number of rooms in the call+maximum number        of streams

As will be discussed in detail below, the priority value of eachconfiguration among a group of configurations for the same combinationof configuration factors (room type+operation mode+number of rooms inthe call+maximum number of streams) is used by the system to determinethe best configuration to use in a telepresence conference. Generally, apriority value of ‘0’ indicates a best configuration and a priorityvalue of ‘1’ indicates a second best configuration.

Table C below shows a configuration table for room type 1, which bydefinition has one active display and one active camera. The record hasa room type value of 1 and the operation mode has a value of 1indicating that in this configuration the room is operating in ‘normal’mode. The “way” value is 2 because this room type can only participatein a conference having 2 rooms. The maximum stream value can only be 1because room type 1 has only one camera. The stream_comb_out value issimply 1 because there is one stream going to the only other room in thecall. The “streams_out” value is “0” because the stream from theleft-most (and only) camera is assigned to the first (and only) otherroom in the call. Likewise, the “streams_comb_in” and “streams_in”values are simply 1 and 0, respectively, because there is only 1incoming video stream and that video stream is displayed on the onlyparticipant display, which is identified with position indicator ‘0’.Lastly, the priority of this configuration is “0” indicating a bestconfiguration. Indeed, there are no other available configurations forthis room. However, as will be seen below, the priority value isemployed where several configuration are available.

TABLE C stream streams stream r type op mode way stream comb out outcomb in streams in priority 1 1 2 1 1 0 1 0 0

Table D below shows a configuration table for a room type 2, which bydefinition has two active displays and two active cameras. As with roomtype 1, room type 2 only has configurations to operate in the ‘normal’mode of operation. Therefore, there are three possible configurationscenarios for room type 2. The first two configurations are for two roomconferences where the maximum streams is set to 1 and 2, respectively.The third configuration is for a 3-room conference where the maximumstreams is set to 1. As with the configuration for room type 1, there isonly one configuration for each unique combination of configurationfactors (room type+operation mode+number of rooms in the call+maximumnumber of streams). Therefore, the priority of each configuration is“0.”

TABLE D stream comb streams stream r type op mode way stream out outcomb in streams in priority 2 1 2 1 1 0 1 0 0 2 1 2 2 2 0, 1 2 0, 1 0 21 3 1 1, 1 0|1 1, 1 0|1 0

Table E below shows a configuration table for a room type 3, which bydefinition has 3 displays and 3 cameras. As above, a room of type 3 canoperate in ‘normal’ mode, therefore the r_type and op_mode forconfigurations in this table are 3, and 1, respectively. Room type 3 mayparticipate in calls having 2, 3 or 4 rooms, and with various maximumstream settings. Therefore, there are seven possible configurations forroom type 3. The fifth and sixth configurations for room type 3 are theonly configurations that have the same configuration factors (roomtype+operation mode+number of rooms in the call+maximum number ofstreams). The priority value for the fifth configuration is ‘0’ becausethis configuration is the best for this combination of configurationfactors. The sixth configuration has a different distribution of videostreams between the other rooms in the call, which is less optimal thanthe distribution in the fifth configuration. Therefore, the sixthconfiguration has a priority value of ‘1’ to indicate that, out of thetwo configuration, the fifth configuration is preferable over the sixth.As described below, the system employs the priority value to obtain themost preferable configuration feasible for a conference.

TABLE E r stream comb streams stream type op mode way stream out outcomb in streams in priority 3 1 2 1 1 1 1 1 0 3 1 2 2 2 0, 1 2 0, 1 0 31 2 3 3 0, 1, 2 3 0, 1, 2 0 3 1 3 1 1, 1 0|1 1, 1 0|1 0 3 1 3 2 2, 1 0,1|2 2, 1 0, 1|2 0 3 1 3 2 1, 2 0|1, 2 1, 2 0|1, 2 1 3 1 4 1 1, 1, 10|1|2 1, 1, 1 0|1|2 0

Table F below shows a configuration table for a room type 7, which bydefinition has 4 displays and 8 cameras. Room type 7 has configurationsabove for both ‘Normal’ and ‘Executive’ modes therefore, there is a setof configurations for room type 7 for op_mode value of ‘1’ and anotherset of configurations for an op_mode value of ‘2.’ As with theconfiguration table for room type 3, for several combinations ofconfiguration factors (room type+operation mode+number of rooms in thecall+maximum number of streams) there are several possibleconfigurations, each of which is given a different priority value. Forexample, the sixth, seventh and eighth configurations in Table F havethe same configuration factors, and have priority values of “0”, “1”,and “2,” respectively, which indicate that of the three configurations,the sixth configuration is most preferable, and the seventhconfiguration is more preferable than the eighth configuration.

TABLE F op stream comb stream r type mode way stream out streams outcomb in streams in priority 7 1 2 1 1 2 1 1 0 7 1 2 2 2 3, 4 2 1, 2 0 71 2 3 3 1, 2, 3 3 0, 1, 2 0 7 1 2 4 4 2, 3, 4, 5 4 0, 1, 2, 3 0 7 1 3 11, 1 2|5 1, 1 1|2 0 7 1 3 2 2, 2 0, 1|6, 7 2, 2 0, 1|2, 3 0 7 1 3 2 2, 10, 2|5 2, 1 0, 1|2 1 7 1 3 2 1, 2 0|1, 2 1, 2 0|1, 2 2 7 1 3 3 3, 1 2,3, 4|5 3, 1 0, 1, 2|3 0 7 1 3 3 1, 3 2|3, 4, 5 1, 3 0|1, 2, 3 1 7 1 4 11, 1, 1 0|1|2 1, 1, 1 0|1|2 0 7 1 4 2 1, 2, 1 0|1, 6|7 1, 2, 1 0|1, 2|30 7 1 4 2 2, 1, 1 2, 3|4|5 2, 1, 1 0, 1|2|3 1 7 1 4 2 1, 1, 2 2|3|4, 51, 1, 2 0|1|2, 3 2 7 1 5 1 1, 1, 1, 1 2|3|4|5 1, 1, 1, 1 0|1|2|3 0 7 2 3X 3, 3 1, 2, 3|4, 5, 6 1, 1 1|2 0 7 2 3 X 1, 1 0|5 1, 3 0|1, 2, 3 1 7 23 X 1, 1 2|7 3, 1 0, 1, 2|3 2

As described above, the “streams_out” field of the configuration tableindicates the position indicator (i.e., Stream ID) of each participantcamera connected to another room in the conference. As set forth above,such position indicators may be, for example, 0, 1, 2 or 3 for an4-camera room, or 0, 1, 2, 3, 4 or 7 for an 8-camera room. Further, the“streams_out” field also indicates to which other room each participantcamera is connected, based on a relative position of the room in asorted order of rooms (the process of sorting the rooms is describedbelow and involves prioritizing and positioning the rooms with respectto one another). For example, the “streams_out” value in row 11 of TableF above is “0|1|2” which indicates that participant camera in position‘0’ (i.e., the left-most camera) is connected to the next room in thesorted order of rooms after the room being configured with the settingsin row 11. Further, camera ‘1’ is connected to the following room in thesorted order and camera ‘2’ is connected to the next following room inthe sorted order. In this process, the sorted order is considered aclosed loop. For example, for the last room in the sorted order, the“next” room in the sorted order is the first room in the sorted order,and so on. As can be appreciated, a non-numerical delineator, such asthe vertical line “|” depicted, may be used to indicate the “next” roomin the sorted order of rooms. Another non-numerical delineator, such asa comma “,” may be used to indicate multiple stream assignments to aparticular room.

Similarly, the “streams_in” field of the configuration table indicatesthe position indicator (i.e., Stream ID, e.g., 0, 1, 2 or 3) of eachparticipant display connected to another room in the conference. Asabove, the “streams_in” field also indicates to which other room eachparticipant display is connected, based on a relative position of theroom in the sorted order rooms. The “streams_in” value in row 11 ofTable F above is “0|1|2” which indicates that participant display inposition ‘0’ (i.e., the left-most camera) is connected to the next roomin the sorted order of rooms after room being configured with thesettings in row 11. Further, display ‘1’ is connected to following roomin the sorted order and display ‘2’ is connected to the next followingroom in the sorted order.

A substantial feature and benefit of the present system is the use ofthe configuration tables such as those above to specify the connectionsbetween rooms in a conference because the configuration tables providean effective means to specify the relative preference among the severalconfigurations and the configuration tables can be efficientlysupplemented or modified as new room types and/or operation modes areadded or modified, and the preferences among several configurations canbe easily modified by changing the priority value in the configurationtables.

The use of configuration tables such as described above does requiresome initial determinations. However such initial determinations do notrequire the system operator to determine the configuration settings forall possible conferences. The initial determinations include: (a)predetermining, for every room type, at least one and preferably aplurality of unique, feasible combinations of configuration factors(i.e., operation mode+number of rooms in the call+maximum number ofstreams), (b) predetermining, for each pre-determined unique, feasiblecombination of configuration factors, at least one and preferably aplurality of unique, feasible permutations of the number of streams thatcan be distributed to and received from every other room in the call,(c) predetermining, for every unique, feasible permutation of the numberof streams that can be distributed to and received from every other roomin the call, stream IDs to be distributed to and received from everyother room in the call (where a stream ID designates a relative positionin which an associated participant camera/display pair is located in acomposite participant display, for example in position 0, 1, 2 or 3,from left-to-light), (d) predetermining, for each unique combination ofconfiguration factors (as determined in (a)) having more than one set ofthe number of streams that can be distributed to and received from everyother room in the call (as determined in (b))+stream IDs to bedistributed to and received from every other room in the call (asdetermined in (c)), a relative preference of each set, (for example withsuch preferences being indicated on a numerical scale, with 0 indicatingmost preferable, and 1 and 2 indicating successively lesser preference).

For example, the configuration table for a room of type 3 as set forthin Table E above has 7 rows of feasible configurations. The initialdeterminations, as determined in (a) above for this room type in resultin 6 unique, feasible combinations of configuration factors (i.e.,operation mode+number of rooms in the call+maximum number of streams).For each of the unique combination of configuration factors in rows 1-4and 7, there is only one set of the number of streams that can bedistributed to and received from every other room in the call (asdetermined in (b))+stream IDs to be distributed to and received fromevery other room in the call (as determined in (c)). Therefore, asdetermined in (d), the priority value for each of these rows is ‘0’,indicating a most preferable configuration, which in this case is theonly feasible configuration. The combination of configuration factors inrow 5 is duplicated in row 6 because, for this combination ofconfiguration factors, there exist two feasible configurations. Theconfiguration of row 5 is more preferable than the configuration of row6. Therefore, the relative preferences of rows 5 and 6 are set to ‘0’and ‘1’, respectively to indicate that row 5 is more preferable.

Participant Camera/Display Pair Assignments

The determinations of the Stream IDs to distribute to and from otherrooms in the conference are chosen to achieve realistic (virtual) eyecontact. To achieve or maximize realistic virtual eye contact among allthe participants of the conference, the system connects the participantdisplays and cameras of each pair to the same remote room.

Generally, the participant display/camera pairs in the highest sortedtelepresence room are assigned to the other rooms in the conferencebeginning from one side (e.g., from left-to-right), in the sorted orderof the other rooms in the conference. Thus, generally, the left-mostparticipant display/camera pairs(s) in the highest sorted room will beassigned to the second-highest sorted telepresence room, and so on.

To further enhance the virtual eye contact among the participants of theconference, the system preferably connects the left-most activeparticipant display/camera pair(s) (and associated codecs) in a giventelepresence room to the right-most active participant display/camerapair(s) (and associated codecs) in the next lower-sorted telepresenceroom. Further, the left-most active participant display/camera pair(s)(and associated codecs) in the lowest-sorted telepresence room arepreferably connected to the right-most active participant display/camerapair(s) (and associated codecs) in the highest-sorted telepresence room.Subsequent assignment of unassigned participant display/camera pairs ineach telepresence room is preferably performed from left to right inorder of priority of the lower-sorted rooms.

If such assignments (for example the assignment between thelowest-sorted telepresence room and the highest-sorted telepresenceroom) would result in an unassigned participant display/camera pairbetween two assigned participant display/camera pairs, the systempreferably alters (e.g., shifts) the assignments such that there existsno unassigned participant display/camera pairs between two assignedpairs.

Initiation of a Teleconference

To initiate a telepresence conference, the user or operator first enterscertain parameters into the system, including:

-   -   1. The customer name;    -   2. The particular customer rooms to be connected;    -   3. The number of participants in each room;    -   4. The identify of a room from which the alternate presentation        will start (if any) and the type of alternate presentation        (e.g., VGA Laptop and/or DVD Video, etc.); and    -   5. The identity of any LVC (ISDN) bridge rooms, if any LVC        systems are to be connected to the conference.    -   6. The maximum desired number of streams between any two room.    -   7. The date & time to initiate the conference, and the duration        or time to terminate the conference.

Determination of Maximum Number of Streams Between Any Two Rooms

With respect to the user input of the maximum desired number of streamsbetween any two rooms, the system preferably provides a drop-down listhaving values of 1 to an absolute maximum number of streams (Xmax),where Xmax is the most streams that may be configured between two roomsin the desired conference. The system also preferably determines adefault value for an optimal number of streams between the rooms, whichis used unless the user overrides the default value by picking anothervalue in the drop-down list.

To determine the absolute maximum number streams (Xmax), the systempreferably employs an algorithm to determine the maximum streams thateach room may receive, sorts each value in descending order and thenchooses the 2^(nd) greatest value.

A first step in this algorithm may be represented by the following:

FOR ALL ROOMS [1. . . N]

Xn=Sn−(N−c);   Formula A

-   -   where c=1 if LVC=1, or c=2 if LVC=0    -   or if LVC>1, c=2−Number of LVC

OR

Xn=(Sn−LCV)−(N−2)   Formula B

-   -   Where LVC=Number of LVC present

END FOR

-   Where N=number of rooms involved in the call. (N must be equal to or    greater than 2)-   Sn=Number of active (i.e., working) screens in room ‘n’ where ‘n’ is    a number between 1 to N.-   Xn=Absolute maximum number of incoming streams for room ‘n’.-   c=An integer as defined above-   LVC=Number of LVC conferences in the call    (Note: A single LVC conference may include participants from several    sites, however, each LCV conference is considered one LVC conference    for purposes of this telepresence system because each LVC conference    requires exactly one display in each telepresence room)

In the second step of the algorithm, the system sorts the list of Xn indescending order (highest-to-lowest). Next, the system select the 2^(nd)element (2^(nd) greatest) in the sorted list as (Xmax). Thus, thedrop-down list will present possible “max” streams from 1 to Xmax

For example, in a 2-way where one room has 4 displays and the other has3 displays, the maximum incoming stream values for each room will beX1=4 and X2=3, and the sorted list is {4, 3}. A 4 stream call ispossible for the first room. However, only a 3 stream call is possiblefor second room, hence the second value 3 is the maximum number ofstreams between the two rooms.

For a 3-way call with 3 rooms having 4 displays in the first room, 3displays in the second room, and 2 displays in the third room, thesorted list will be {3, 2, 1}. A 3-way call would never be possiblebecause only the first room can handle 3 extra streams, hence the secondlargest value 2 will be the maximum possible stream value.

For purposes of this determination, the sorted list may containduplicate values. In other words, duplicate values are not removed fromthe sorted list for this procedure. However, duplicate values arepreferably removed from the drop-down list presented to the user, asdescribed below. For example, for a 3-way call between 3 rooms, wherethe first and second rooms have four displays and the third room has 2displays, the sorted list would be {3,3,1}. The second value is 3 whichis the correct possible maximum value for max-stream number.

It is important to note that the maximum possible number of streamsbetween any two rooms (Xmax) is not necessarily the highest stream valuecommon to all rooms, which is called optimal (or default) maximum streamvalue. The algorithm to determine the optimal maximum stream firstdetermines the number of displays in the room having the least number ofdisplays, and then determines how many streams that room can receivefrom each other (non-LVC) room in the call.

Specifically, in a first step, the system steps through each room inconference, to determine the room with least number of displays, andstores the number of displays in that room (Smin). Then the system usesthe following formula to determine the optimal number of maximum streams(Xopt):

Xopt=INT[(Smin−LVC)/(N−1)]

-   -   where LVC is the number of LVC conferences in the call    -   N is the total number of rooms in the call, including any LVC        conference    -   Xopt is a value rounded down to the next integer

As discussed above, the optimal maximum stream value is used by thesystem as the maximum number of streams to use in the conference unlessthe user overrides the setting by choosing another maximum stream valuebetween 1 and the absolute maximum stream value, via the drop down listduring initial entry of the conference-variable information.

Preliminary Compliance and Diagnostic Checks

After completion of the entry of conference-variable information, thesystem will preferably make initial determinations to ensure that thedesired conference configuration is feasible. If the desired conferenceconfiguration is not feasible, the system preferably displays anappropriate error message to the user.

Specifically, to ascertain whether a conference may be configuredaccording to the user input, the system preferably performs certainpreliminary compliance and diagnostic checks. A first preliminarycompliance check is related to the number of active cameras in eachtelepresence room. To establish a desired conference, each telepresenceroom (i.e., non-LVC conference) in the conference must have a sufficientnumber of active (i.e., operable) participant cameras to capture aunique image for each other room in the conference (including any LVCconference). This ensures that each remote room in the conference(including any LVC conference) can be sent a unique viewing angle. Thus,the number (C) of active participant cameras in each telepresence roommust be greater than or equal to the number of rooms in the conference(N) minus one, plus any LVC conference in the call. [C>=N−1+LVC].

The system may include automated or manual diagnostic routines andprocedures to determine at this point in time, whether any of theparticipant cameras are inoperable or otherwise unavailable for use inthe teleconference. The system may consider a camera to be inoperable ofunavailable if an associated display is inoperable or unavailable.

It can be appreciated that the above requirement is not necessary forLVC conferences in a conference because such rooms have only one cameraand thus transmit the same image to all other rooms in a conference.

A second preliminary compliance check is related to the number of activedisplays in each telepresence room. Each telepresence room (i.e.,non-LVC conference) in the conference must have a sufficient number ofactive participant displays to display at least one incoming videostream from all of the other rooms in the conference. Ordinarily, foreach telepresence room, each active participant display will displayonly one incoming video stream. Thus, for such each telepresence room inthe conference, the number of active participant displays in the room(D) must be greater than or equal to the number of telepresence rooms onthe conference (N) minus one, plus any LVC conference [D>=N−1+LVC].

As with cameras, the system may include automated or manual diagnosticroutines and procedures to determine at this point in time, whether anyof the participant displays are inoperable or otherwise unavailable foruse in the teleconference. The system may consider a display to beinoperable of unavailable if an associated camera is inoperable orunavailable.

A third preliminary compliance check is related to alternativepresentation or a lectern presentation. When such an alternative (orlectern) presentation is desired, one telepresence room is designated asthe initiating room, meaning that the alternative (or lectern)presentation will be generated (at least initially) from the initiatingroom. A necessary requirement for such a conference is that eachnon-initiating, telepresence room (i.e., non-initiating, non-LVC room)must have an active alternative presentation display. The system mayinclude automated or manual diagnostic routines and procedures todetermine whether any desired alternative presentation equipment ordisplays are inoperable or otherwise unavailable.

Each LVC may not require a separate alternative presentation displaysince, with some systems, such an alternative presentation will bedisplayed in one of the “boxes” of the standard array of boxes in asingle LVC display (i.e., a “Hollywood Square” format).

If any of the above three conditions are not met for any telepresenceroom in the conference, the system preferably terminates the scenarioalgorithm and generates an error message, such as “Cannot set up thecall because of insufficient equipment in Room {Name},” where {Name} isreplaced by a predetermined name of the room which does not meet therequirements. The system may also identify the missing equipment.Conversely, if all of the above conditions are met, then the desiredconference configuration is feasible and the system moves to the nextsteps to configure the conference.

The system may perform other diagnostic checks at this point, such as toconfirm that all necessary audio-video switching equipment located atthe conference sites or at a central location are available andoperable, and that the necessary network connections and bandwidth areavailable and operational.

Determination of Optimal Conference Configuration

After the preliminary compliance checks and preliminary diagnosticchecks, the system preferably proceeds to configure the desiredconference. The end result of the configuration process is one or moreconference configuration scripts which are transmitted to equipment atthe central network location and the sites of the various teleconferencerooms to interconnect the rooms for the teleconference.

Prioritization of Rooms in the Room List

As a first step in the configuration of the teleconference, the systempreferably creates a room list consisting of the desired rooms enteredby the user during entry of the conference-variable information. Thesystem preferably performs a first sort of the room list according topredetermined prioritization criteria. Prioritization of the rooms isnot absolutely necessary, but there are scenarios when prioritizing therooms provides optimal vectoring and resource utilization for the roomwith highest priority. By prioritizing the rooms the system ensures thatrooms with higher priority get improved vectoring and resourceutilization.

Prioritization of the rooms can be done based on many differentparameters. The system can assign the highest priority to a room withthe highest number of participants, the highest number of displays orcamera, or even some combination of the number of participants, displaysand camera. The prioritization algorithm may be adjusted as and whenrequired because changes would not require substantial changes to theremainder of the system.

Below is a representation of a prioritization logic which is basedprimarily on (display number+camera number) and secondarily on(participant number) in each room. The algorithm implements a sortinglogic. As can be appreciated, various sorting algorithms may be usedinstead, like bubble sort, merge sort, heap sort or quick sort, or othersorting algorithms.

START FOR each room indexed ‘I’ in the room-list from 1 to N = size ofthe list  FOR each room indexed ‘J’ in the room-list from N down-to(‘I’+1)   IF (display no. + camera no.) of room at index (‘J’−1) ==(display   no. + camera no.) of room at index (‘J’) THEN    IF(participant no.) of room at index (‘J’−1) >= (participant    no.) ofroom at index (‘J’) THEN     swap the rooms between positions (‘J’) and(‘J’−1)    END IF   ELSE    IF (display no. + camera no.) of room atindex (‘J’−1) >    (display no. + camera no.) of room at index (‘J’)THEN     swap the rooms between positions (‘J’) and (‘J’−1)    END IF  END IF  END FOR END FOR STOP

After completion of the prioritization sort of the room list, the systempreferably adds any LVC conferences to the end of the room list.

Positioning of Rooms in the Room List

The system may also perform a second sort of room in the room list toposition the rooms in the room list according to a predeterminedposition weight, or may perform this second sort instead of the firstsort. This sort is intended to replicate the positions in which remoterooms are displayed on multiple displays within a given telepresenceroom so that repeat teleconference participants become accustom to andcomfortable with the positions of the other rooms. It has been foundthat such replication enhances the simulation of an in-person meeting.

For example, consider the scenario where a company using theteleconference system has telepresence rooms in New York, London, Parisand Tokyo, each of which has 4 displays. In a call with New York, Londonand Tokyo in the morning at 10:00 AM, participants in the room in NewYork may see participants from London in the left two screens andparticipants from Tokyo in the right two screens. Later, in a call inthe evening at 4:00 PM if the participants in London do not appear inthe left two screens then it may lead to confusion for the participantswho also participated in the first call in the morning. To eliminatethis confusion for the users and to give users a consistent view everytime they participate in a conference, the system may employ thepositioning algorithm to fix positions with each room.

The positioning is another sorting algorithm based on a predeterminedposition weight associated with every room. As can be appreciatedvarious sorting algoritiuns may be used to accomplish the sort. Onesuitable sorting algorithm is set forth below.

START FOR each room indexed ‘I’ in the room-list from 1 to N = size ofthe list  FOR each room indexed ‘J’ in the room-list from N down-to(‘I’+1)   IF (positioning weight) of room at index (‘J’−1) >=(positioning   weight) of room at index (‘J’) THEN    swap the roomsbetween positions (‘J’) and (‘J’−1)   END IF   END FOR END FOR STOP

Rooms are sorted in order depending on their weight value from lowest tohighest. Consider 5 rooms and their weight: London—1, Paris—2, NewYork—3, Tokyo—4, and Sydney—5. The important thing here is for in anycall involving New York, the participant may want to see London andParis in the left, and Tokyo and Sydney in the right. For calls belowthey will be ordered, from left to right:

-   -   London, Paris, NY    -   London, NY Tokyo    -   Paris, NY, Sydney    -   London, NY, Sydney    -   Paris, NY, Tokyo    -   London, Paris, NY, Sydney    -   Paris, NY, Tokyo, Sydney    -   London, Paris, NY, Tokyo, Sydney

In no case will Paris be on the left most side if London is present incall for the NY participants. As shown, the weight values are sequentialintegers, however other sequential or non-sequential values may be usedto indicate relative positional weighting of rooms. Further, the valuesmay be sorted from highest to lowest, if the pre-detennined positionalweight values are arranged in reverse order.

Determine Distribution of Streams Between Each Room in Call

Upon completion of the prioritization and positioning procedures, thesystem proceeds to determine the distribution (and number) of streamsbetween each room in the call. To accomplish this procedure, the systememploys an iterative, reverse hill-climbing algorithm using theconfiguration tables set forth above to determine the most optimal,feasible configuration for the teleconference.

In selecting configuration setting for a particular room from theappropriate configuration table, the system first chooses the mostoptimal configuration for the room, which is identified by theconfiguration with the lowest priority value of ‘0’. As can beappreciated, other values, or reversed values can be used to indicaterelative priority. If the selected configuration is not feasible, thesystem then selects then next lower optimal configuration and attemptsto configure the conference. The system continues with the iterativeprocess, which sometimes requires backing out settings made for previousrooms to arrive at a workable configuration for all rooms. The algorithmemployed by the system may be described as a reverse hill-climbingalgorithm because the system knows of and attempts to use the mostoptimal configuration first and then, if necessary, backs down tosuccessively lesser optimal configurations until workable configurationsare achieved for all rooms in the call.

A pseudo code for the algorithm, including the prioritizing andpositioning procedures is set for the below.

1. START 2. Get list of rooms from DSMConference as roomList. 3.PRIORITIZE the rooms in roomList. [as described above] 4. If theconference using LVC add LVC conference at the end of the roomList. 5.POSITION the rooms in roomList. [as described above] 6. Initializealgorithm specific data for all rooms in the room list. 7. FOR each roomin the roomList LOOP UNTIL all rooms exhausted or vectoring SUCCESSFUL.a. Pickup the current room as room identified by current roomIndex. b.Get configuration(s) for the current room for the combination ofconfiguration factors (room type + operation mode + number of rooms inthe call + max streams) c. Identify the next configuration to process inthe current context. d. IF there is no configuration left to processTHEN i. ROLLBACK any stream distribution done last time. ii. START thisprocess from step 7(a) with the PREVIOUS room. - RECURSIVE CALL e. ELSEi. DISTRIBUTE the streams of this configuration of this room to ALLOther rooms. ii. IF stream distribution SUCCESSFUL THEN 1. START thisprocess from step 7(a) with the NEXT room. - RECURSIVE CALL 2. Setdistribution success of this room as TRUE iii. ELSE 1. ROLLBACK anystream distribution done last time. 2. START this process from step 7(c)with the CURRENT room. - RECURSIVE CALL iv. END-IF f. END_IF 8.END-FOR_LOOP 9. IF all rooms stream distribution status is successfulthen we achieved vectoring successfully else vectoring not possible forthe given conference. 10. Conclude and generate stream table from thestream distribution on SUCCESS - [Logic described below] END

Step 7(e)(ii) is “IF stream distribution SUCCESSFUL THEN START thisprocess from step 7(a) with the NEXT room. —RECURSIVE CALL. In this step“Successful” means that, for the present room, it was possible to assignthe number of outgoing and incoming streams designated in thestream_comb_out and stream_comb_in fields of the current configurationto and from the other rooms in the call. In other words, in the presentroom and other rooms, the number of active displays and cameras issufficient to accept the designated number of outgoing and incomingstreams, in addition to any previous designations.

Step 7(c) states “Identify the next configuration to process in thecurrent context.” In this step, the system selects the next lesseroptimal (or next lesser preferable) configuration for the current room,which configuration has not yet been used in a previous attempt toconfigure the current room, unless the system has rolled back andchanged the configuration of a previously configured room, in which casethe system may re-try previously tried configurations for a subsequentroom. However, the system does not attempt to re-use a configuration ifall configurations for the room next in the room list failed (i.e.,during a roll back).

As indicated in step 7(e)(iii)(1), the algorithm rolls back any streamdistributions made for the current room if all distributions identifiedin the current configuration for the current room are not successful.However, if all possible configurations fail for the current room, thenthe system rolls back the distributions made for the previous room(7(d)(i)) and tries the next lesser optimal configuration for theprevious room (7(c)). Assuming the next lesser optimal configuration wassuccessful for the previous room, the system then proceeds to re-attemptstream distribution for the previously-failed room (now the currentroom) starting with the most optimal configuration, even if the mostoptimal configuration failed previously. In this instance, the systemwill attempt to use a previously failed configuration because theconfiguration for at least one previous room has changed since the prioriteration for the current room. Therefore, a configuration that failedpreviously for the current room may be feasible now.

Generation of the Stream Table

The distribution of streams among the rooms in the conference asdescribed above determines the number of streams that pass between eachroom in the conference. To determine which displays and cameras of eachof the rooms are connected together, the system proceeds to generate astream table, which is a two-dimensional array having N columns and Mrows, where N is equal to the number of rooms in the call, and M isequal to the aggregate number video streams in the conference, a videostream being a stream of video data from a camera, which is presented ona display.

Referring to FIG. 5, a 3-way conference between a first room having 4cameras and displays and second and third rooms each having 2 camerasand displays is depicted along with a stream table suitable for theconference configuration. In the schematic, the first room in the sortedroom_list is positioned at the bottom of a virtual conference circle(i.e., at the 6 o'clock position), with the subsequently sorted roomsproceeding clockwise around the virtual circle. The cameras of the firstroom are designated as 0C0, 0C1, 0C2 and 0C3, indicating the ‘0’ roomand the position of the associated camera/display pair from left toright (as viewed by participants in that room) starting with the 0position.

The streams passing between the rooms are indicated by the directionalarrows, which start at a camera and ultimately end at a display. Thestream table array on the right side of the figure shows the connectionsbetween the various cameras and displays in the conference in tabularform, as specified by the configuration settings chosen by the systemduring the configuration processes. In the normal mode of operation,cameras and displays are connected in reciprocal fashion. Therefore,since in the first row of the stream table, camera 1 of room 0 isconnected to display 1 of room 1, then camera 1 of room 1 is connectedto display 1 of room 0, as indicated by row 4 of the table. However, itshould be noted that in other modes of operations, such as the‘executive’ mode, the reciprocal connections may not always exist.

Referring to FIGS. 6 and 7, 4- and 5-way conferences are depicted inschematic form, along with the corresponding stream tables as generatedby the system. As can be appreciated, the stream table is a suitablemeans to set forth the necessary connections in conferences of many (orany number of) rooms, and among rooms having various numbers ofcamera/display pairs.

As mentioned above the connections between cameras and displays in aconference is determined by information in the streams_out andstreams_in field of the configurations chosen by the system during theconfiguration processes. For example, in Table E, the 5^(th)configuration is suitable for a room type 3 (3 displays, 3 cameras),operating in normal mode, in a call with 3 rooms total (i.e., 2 otherrooms), with a maximum stream value of 2. The information in thestreams_out field for this configuration is “0,1|2”, which indicatesthat cameras 0 & 1 (i.e., the left-most, and next camera to the right)are to be connected to the next room in the room list, which is the nextroom clockwise in the virtual circle, and camera 2 is to be connected tothe room after the next room in the room list, which is two rooms awayclockwise in the virtual circle from the present room, with the verticalbar indicating the separation between the rooms. The streams_ininformation provides similar information for streams coming in to theroom. In particular, the streams coming in from the next room clockwisein the virtual circle are to be assigned to display numbers ‘0’ and ‘1,’and the stream coming in from the room after the next room clockwise inthe virtual circle is to be connected to display number ‘2’.

As a comparison, the 4^(th) configuration in table E is similar to the5^(th) configuration however the maximum stream value is 1, not 2.Therefore there can only be one stream passing between the rooms.Accordingly, the streams_out information is “0|1” which indicates thatcamera ‘0’ is to be connected to the next room clockwise in the virtualcircle and camera ‘1’ is to be connected to the room after the next roomclockwise in the virtual circle.

Below is a stream table generation algorithm suitable for generating astream table as depicted, from a plurality of configuration chosen bythe system.

START FOR each room in the room-list identified by roomIndex. Get the‘out-streams’ for the room at hand separated in parts.  Example:out-stream = [0|1, 2|3].  Part [1, 1] = 0  Part [2, 1] = 1, Part [2, 2]= 2  Part [3, 1] = 3   FOR each Part-list after break character ‘|’identified by   partIndex1    FOR each Part in part-list after breakcharacter ‘,’    identified by partIndex2     1.streamTableRow[roomIndex] = ‘C’ + Part [     partIndex1, partIndex2]    2. nextRoomIndex = generate next room index from     currentposition     3. Get the ‘in-stream’ for the room identified by    nextRoomIndex separate in parts.     4. inStreamIndexPart1 =in-streams part length − 1 −     partIndex1     5. inStreamIndexPart2 =in-streams part[partIndex1]     length − 1 − partIndex2     6.streamTableRow[nextRoomIndex] = ‘D’ + Part     [inStreamIndexPart1,inStreamIndexPart2]     7. insert ‘streamTableRow’ into stream table   END FOR   END FOR  END FOR  END

Configuration Script Generation and Transmission

After generating the stream table as set forth above, the systempreferably generates scripts to configure and interconnect theteleconference equipment such as to configure the seating positionindicator lights and to configure codec teleconference activation anddeactivation connector application/systems at each of the telepresenceroom sites to effectuate the assignments. Preferably such configurationscripts are in a format suitable to be received and executed byapplication/systems and are transmitted to the application/systems overthe network (such as in XML format). Such a configuration script forparticipant video stream assignments (streams.xml) may be in thefollowing form:

-----------------------------StartStreams.xml----------------------------- <?xml version=“1.0”encoding=“UTF-8”?> <Messagexmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”>  <Stream>  <Encoder>    <IPAddress>1.2.2.3</IPAddress>   <EncoderName>london0</EncoderName>    <SlotId>1</SlotId>   <PortId>0</PortId>    <CopyNum>0</copyNum>    <UDPPort>4443</UDPPort>  </Encoder>   <Decoder>    <IPAddress>1.2.2.3</IPAddress>   <DecoderName>India1</DecoderName>    <SlotId>1</SlotId>   <PortId>0</PortId>    <CopyNum>0</copyNum>    <UDPPort>5000</UDPPort>  </Decoder>  </Stream>  <Stream>   <Encoder>   <IPAddress>1.2.2.4</IPAddress>    <EncoderName>london1 </EncoderName>   <SlotId>1</SlotId>    <PortId>0</PortId>    <CopyNum>0</copyNum>   <UDPPort>4443</UDPPort>   </Encoder>   <Decoder>   <IPAddress>1.2.2.3</IPAddress>    <DecoderName>India0 </DecoderName>   <SlotId>1</SlotId>    <PortId>0</PortId>    <CopyNum>0</copyNum>   <UDPPort>5000</UDPPort>   </Decoder>  </Stream>  <Stream>   <Encoder>   <IPAddress>1.2.2.6</IPAddress>    <EncoderName>london2 </EncoderName>   <SlotId>1</SlotId>    <PortId>0</PortId>    <CopyNum>0</copyNum>   <UDPPort>4443</UDPPort>   </Encoder>   <Decoder>   <IPAddress>1.2.2.8</IPAddress>    <DecoderName>India2 </DecoderName>   <SlotId>1</SlotId>    <PortId>0</PortId>    <CopyNum>0</copyNum>   <UDPPort>5000</UDPPort>   </Decoder>  </Stream> </Message>-----------------------EndStreams.xml------------------------------------

If a laptop alternate presentation is in the conference, the systempreferably generates and transmits a laptop alternate presentationconfiguration script, which may, for example, transmitted to Lara echopresentation connector applications. Such a laptop alternatepresentation configuration script (presentation.xml) may be in thefollowing form:

-----------------StartPresentaion.XML----------------------------------- <? xml version=“1.0”encoding=“UTF-8”?> <Displayxmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”>  <VGA>  <SourceIPAddress>12.23.345.4 </SourceIPAddress>  <DestinationIPAddress> 12.34.45.6</DestinationIPAddress>  <DestinationIPAddress> 12.34.45.7</DestinationIPAddress>  <DestinationIPAddress> 12.34.45.8</DestinationIPAddress>  </VGA></Display> --------------------------EndPresenatation.xml--------------------------

If a DVD alternate presentation is in the conference, the systempreferably generates and transmits a DVD alternate presentationconfiguration script. Such a DVD alternate presentation configurationscript may be in the following form:

--------------StartDVDPresentation.xml----------------------------------- <?xmlversion=“1.0” encoding=“UTF-8”?> <Dvd>  <Stream>   <Encoder>   <IPAddress>1.2.2.3</IPAddress>    <EncoderName>london0</EncoderName>   <SlotId>1</SlotId>    <PortId>0</PortId>    <UDPPort>4443</UDPPort>  </Encoder>   <Decoder>    <IPAddress>1.2.2.5</IPAddress>   <DecoderName>India1</DecoderName>    <SlotId>1</SlotId>   <PortId>0</PortId>    <UDPPort>5005</UDPPort>   </Decoder>  <Decoder>    <IPAddress>1.2.2.8</IPAddress>    <DecoderName>Singapore1</DecoderName>    <SlotId>1</SlotId>    <PortId>0</PortId>   <UDPPort>5000</UDPPort>   </Decoder>   <Decoder>   <IPAddress>1.2.2.10</IPAddress>    <DecoderName>boston </DecoderName>   <SlotId>1</SlotId>    <PortId>0</PortId>    <UDPPort>5000</UDPPort>  </Decoder>  <Stream> </Dvd> -------------EndDVDPresentation.xml------------------------------------

Preferably, each assignment is recorded in a log file for debugging.

If lectern alternate presentation is in the conference, the systempreferably generates a lectern alternate presentation script, as setforth below. The room that is selected as the source will have a streamconnected from the source encoder to each destination decoder, and eachdestination encoder will connect to the source decoder. The encoder anddecoder that can be used will have active status as ‘L’.

----------------Example Lectern XML-------------------------------------<?xml version=“1.0” encoding=“UTF-8”?> <Lectern>  <Stream>   <Encoder>   <IPAddress>10.254.230.13</IPAddress>    <Name>CameraFour</Name>   <SlotId>1</SlotId>    <PortId>5</PortId>    <UDPPort>5040</UDPPort>  </Encoder>   <Decoder>    <IPAddress>10.254.230.45</IPAddress>   <Name>DisFour</Name>    <SlotId>1</SlotId>    <PortId>5</PortId>   <UDPPort>5040</UDPPort>   </Decoder>  </Stream>  <Stream>   <Decoder>   <IPAddress>10.254.230.13</IPAddress>    <Name>CameraFour</Name>   <SlotId>1</SlotId>    <PortId>5</PortId>    <UDPPort>5040</UDPPort>  </Decoder>   <Encoder>    <IPAddress>10.254.230.45</IPAddress>   <Name>DisFour</Name>    <SlotId>1</SlotId>    <PortId>5</PortId>   <UDPPort>5040</UDPPort>   </Encoder>  </Stream> </Lectern>--------------------------------end of LecternXML-----------------------

Referring to FIG. 8, processing of the configuration scripts results inappropriate interconnections between switching and codec equipmentemployed in the teleconference, which equipment may be located at thesites of the various rooms in the conference or may located at a centrallocation. For example, in a conference having three telepresence rooms10, 12 and 14, with displays 16, 17, 18, 19, 20, 21, 22, 23, 24 and 25and cameras 26, 27, 28, 29, 30, 31, 32, 33, 34 and 35, and one LVC room15, the system may interconnect certain camera and display codecs(labeled “0”, “1”, “2” and “3”) to effect the desired configuration. Ifan LVC 15 is in the call, the LVC may be connected to a camera/displaypair in each room. As can be appreciated, the interconnection depictedin FIG. 8 are but one possible configuration and may not be the mostoptimal configuration.

Post-Configuration Testing and Diagnostics

After configuration of the teleconference and before conducing theteleconference, the system may perfonn various automated, semi-automatedor manual diagnostic procedures to confirm operation of all of thenecessary equipment in the room. Upon failure of a necessary piece ofequipment, such as a camera, display or codec, it may be necessary toreconfigure the teleconference. In this case the system may re-run theconfiguration procedures beginning with sorting the rooms according topriority, which may be based in part on the existence of certainoperable equipment. Alternatively, the system may re-run theconfiguration beginning with the procedure to determine streamdistributions between the room in the conference, preferably startingwith the first room in the room list.

Performance and Termination of Telepresence Conference

Upon the completion of the post-configuration diagnostics andre-configuration (if necessary), the system may activate theteleconference, at a predetermined time or upon receipt of a command orcommands from the user. The teleconference remains active until apredetermined termination time, if one has been established, or uponreceipt of a command or commands from a user to terminate theconference.

The special purpose computers and/or general purpose computers locatedat the site of the system operator (e.g., the NOC) preferably performall (or a majority) of the processing to configure each teleconferenceand to interconnect the teleconference equipment. Specifically, the datatables referred to above (i.e., the room, mode and configuration tables)are preferably computer data tables stored in or accessible by thecomputers of the system operator. During initiation of a teleconference,the conference-specific data is preferably entered into computer dataterminal connected to the computers of the system operator, such as adata-entry terminal or computer located at the site of the systemoperator or at the site of one of the rooms participating in theconference, or at another location. During the entry of theconference-specific data, the computers of the system operatorpreferably perform the steps of determining the maximum number andoptimal number of streams between rooms in a conference, and providesuch information to the data entry terminal. Subsequent to the entry ofthe conference-specific data, the computers of the system operatorpreferably perform the steps of preliminary compliance and diagnosticchecks, prioritization & positioning of rooms in the room list,determining the distribution of streams between each room in theconference, generation of the stream table, generation & transmission ofthe configuration scripts to the teleconference equipment,post-configuration testing and diagnostics, and optionally terminationof the conference.

Remote Gateway Device

Referring to FIG. 8, to accommodate an LVC conference into a conferencewith telepresence rooms, each telepresence site or room may include aLVC-specific codec 15 suitable to interconnect a camera and display pair(e.g., pair 23, 33) in each telepresence room with the equipment orsystem of one or more predetermined providers of legacy video conferenceservices. However, instead of such an LVC-specific codec in each room orsite, the system preferably includes a gateway device (not shown)located remotely from at least one of the telepresence rooms in theconference which is operable to simultaneously interconnect a pluralityof external sources of audio and/or video to the telepresenceconference, provided such external sources comply with at least one of aplurality of predetermined audio/video standards, such as H.323, SIP, orthe like, and/or comply with a predetermined standard of a provider oflegacy video conferencing services.

The gateway device is operable to interconnect and distribute suchexternal sources to one of the (non-LVC-specific) telepresenceconference codecs located at the site or room of each telepresence roomin a telepresence conference, and is operable to perform any necessarytranslations of audio/video data or transmission protocol necessary tointerconnect such external sources to such telepresence codecs. Thus,the remote gateway device eliminates the need for each telepresence siteor room to have an LVC-specific codec.

Enhanced Directional Audio

To enhance the simulation of an in-person conference, each participantspeaker associated with each display/camera pair is preferably locatedadjacent the associated display/camera pair. The assignments made abovesimultaneously assign each participant speaker to the remote roomassigned to the associated (adjacent) display/camera pair, via theassociated Audio/Video codec. Thus, conversation and other audiogenerated at a remote room will be reproduced in the local room adjacentthe assigned display/camera pair.

More specifically, to further enhance the simulation of an in-personconference in telepresence rooms having composite participant displayscomprising multiple individual displays and having a separateparticipant speaker associated with each of the multiple participantdisplays, sound produced by conference participants in other rooms ispreferably reproduced only by (or substantially only by) the participantspeaker associated with the participant display in which the participantappears. Thus, the attention of participants is instinctively andautomatically drawn to the direction of the participant display in whichthe video image of a person speaking is presented even without visualclues as to which one of the remote participants is talking. Suchone-to-one spatial association of video and audio is particularlyeffective in conferences where 3 or 4 participant displays are active ina given room.

As described above, the system preferably directs participants topredetermined seating locations by automatically activating the seatingposition markings 110 (and most preferably the seating position lights)according to a predetermined algorithm processed by the computers of thesystem operator, which algorithm is based on the number of participantspresent in the room. Alternatively or additionally, the system coulddirect participants to seating locations via a display or readoutlocated in the telepresence room. In general, the system directs theindividuals to fill the conference table from the center out. Forexample, if the conference table includes eight seating locations(numbered 1 through 8, from left to right facing the participantdisplays) and one or two persons are participating in the conference inthat room, the system preferably directs the participant(s) into one ofseating locations 5 & 6 (on either side of the center line 100).Alternatively, the system could direct the participant(s) to otheradjacent seating locations such as numbers 4 & 5, or 6 & 7. If three orfour persons are in the room, the system preferably directs theparticipants to seating locations 3, 4, 5 & 6. Likewise, if seven oreight participants are in the room, the system directs the participantsto seating locations 1 through 8. Thus, the system knows the seatinglocations of all of the participants in each room.

The system preferably automatically activates or deactivates participantmicrophones depending on whether a participant has been directed to aseating locating associated with the particular microphone. Preferably,each participant microphone 40 in the telepresence conference room isassociated with a pair of (two) predefined seating locations 108 and islocated substantially equidistant to each of the pair. Each participantmicrophone 40 is preferably a cardioid microphone. The area ofconversational sensitivity of each participant microphone 40 to ordinaryconversation-level sound preferably only encompasses the two predefinedseating locations 108 with which each participant microphone 40 isassociated. As can be appreciated, for each microphone, the physicalpositioning and the boundary of the area of conversational sensitivityserves to substantially limit the amount of conversation-level soundpicked up by each participant microphone 40 to that conversation-levelsound generated by the participants located in the pair of seatinglocations associated with each participant microphone 40. For example,in a typical telepresence conference room having eight seating locationsnumbered 1 through 8 (from left to right), participant microphonesnumbered 1 through 4 (from left to right) may be associated with seatinglocations 1 & 2, 3 & 4, 5 & 6 and 7 & 8, respectively.

Where only one video stream is sent from a first room in a telepresencecall to a second room, all participants in the first room appear on oneparticipant display in the second room and all of the sound produced inthe first room (and picked up by the active participant microphones inthe first room) is reproduced in the second room preferably only by theone participant speaker associated with the participant display assignedto the first room. The audio signal associated with such one videostream consists of a combination (i.e., sum) of the signals from all ofthe active participant microphones in the first room.

Where multiple video streams are exchanged between rooms in atelepresence conference the multiple audio streams are also exchanged.For example, where two video streams are sent from a first room to asecond room, a first video stream is generated by a first participantcamera capturing a first group of participants in the first room and asecond video stream is generated by a second participant camera in thefirst room capturing a second group of participants. The first videostream (and first group of participants) is displayed on a firstparticipant display in the second room. Likewise, the second videostream is displayed on a second participant display in the second room.To provide enhanced directional audio in such circumstances, the systempreferably isolates the audio for each stream. Specifically, the soundproduced by the first group of participants is preferably reproduced inthe second room only by (or substantially only by) the participantspeaker associated with the first participant display. Likewise, thesound produced in the first room by the second group of participants ispreferably reproduced in the second room only by (or substantially onby) the participant speaker associated with the second participantdisplay.

A first set (of one or more) participant microphones is associated withthe seating locations of the first group of participants in the firstroom, as described above. Likewise, a second set of microphones (of oneor more) is associated with the seating locations of the second group ofparticipants in the first room. For example, if the first group ofparticipants consists of four persons, the first set of participantmicrophones may consists of two microphones.

The system determines the number of streams to be exchanged between eachroom in the conference depending on a number of factors. For eachoutgoing video stream, going from the first room to a second room, thesystem determines the seating locations to be encompassed in the videostream and configures the participant camera generating that videostream accordingly, which configuration may include adjusting a pan,tilt and/or zoom setting of the participant camera, a position of thecamera along a track, and/or a digital crop of the captured images. Forexample, if one video stream is to be sent from a first room to a secondroom, the system configures a single participant camera in the firstroom to capture all of the participants in the first room. Further thesystem creates the audio stream associated with such single video streamby summing the signals of all of the (or more preferably, all of theactive) participant microphones in the first room. Such summing ispreferably implemented by equipment located in the room or site of eachtelepresence room, for example by the 16×16 switch 39, or a similardevice. Further, such summing may be implemented at a central locationor the location of the second (receiving) room, or a combination oflocations.

If two video streams are to be sent from the first room to the secondroom, the system uses two participant cameras to generate first andsecond video streams. The system configures the first participant camerato capture a first group of participants (in a first group ofpredetermined seating locations) and configures the second participantcamera to capture a second group of participants (in a second group ofseating locations). Further, the system creates the audio streamassociated with first video stream by summing the audio signalsgenerated by the participant microphones associated with the first groupof seating locations. Likewise, the system generates the audio streamassociated with second video stream by summing the audio signalsgenerated by the microphones associated with the second group of seatinglocations. Again, such sunuming may be implemented locally (in the firstroom) and/or remotely (in a central location or in the second room), orin several locations. Where more than two video streams are sent toanother room, a similar process is used to segregate the audio signalsassociated with each video stream.

The system employs the configuration information contained in the streamtable to effect the connections of the audio streams to the appropriateparticipant speakers in the conference. Preferably, after the systemsums the audio signals of a group of seating locations (if summing isnecessary), the system connects the audio stream to a participantspeaker associated with the participant display on which the associatedvideo stream is displayed. More particularly, the proper room number tosend the audio stream is identified by the row of the stream table andthe proper participant speaker is identified by the correspondingposition indicator of the associated participant display, since theposition indicator of the associated participant speaker is the same asthat of the associated display. Therefore, the conversation of theparticipants displayed in each participant display is reproduced inclose proximity to the images, which provides for a realistic andcompelling spatial association of video and audio.

As described above, an echo-canceling device may be employed to minimizeor eliminate echoes and feedback. Preferably, each site or room includesa multi-input and multi-output echo-canceling device which is remotelyconfigurable by a central control. The signals being sent to theparticipant speakers (or any speakers) in a room are also fed to theecho-canceling device (either as separate signals or a summed signal) asa base reference signal or signals in order to remove any echo and/orfeedback in the signals generated by the active participant microphones.The processed microphone signals are then passed through theecho-canceling device and distributed to other rooms in the conference.The echo-canceling device may be similar to the echo-canceling device 39described above; however it preferably includes multiple inputs andoutputs and is remotely configurable to interconnect such inputs andoutputs in various combinations, or in any permutation of inputs andoutputs.

1. A method of conducting an audio-visual conference among a number ofconference rooms comprising: storing predetermined conferenceconfiguration data in more or more computer data tables in a computersystem; said conference configuration data including information for aplurality of room types, where each room type has a unique combinationof active participant displays and active participant cameras; for atleast one room type, said conference configuration data has a pluralityof unique, feasible combinations of configuration factors, where saidconfiguration factors include, a number of rooms in the conference, anda maximum number of streams to be distributed to another room in theconference, for at least one of said plurality of unique, feasiblecombinations of configuration factors for said one room type, saidconference configuration data includes a plurality of unique, feasiblepermutations of the number of streams that can be distributed to andreceived from every other room in the conference; for each unique,feasible permutation of the number of streams that can be distributed toand received from every other room in the call, said conferenceconfiguration data includes stream IDs to be distributed to and receivedfrom every other room in the conference, where a stream ID designates aphysical position in which an associated participant camera andparticipant display pair is located in a composite participant display;for each unique combination of configuration factors having more thanone set of (i) the number of streams that can be distributed to andreceived from every other room in the conference and (ii) stream IDs tobe distributed to and received from every other room in the call, saidconference configuration data includes a relative preference of eachset; receiving at said computer system conference variable dataincluding identities of a plurality of included conference rooms to beinterconnected in a conference, and including a desired maximum numberof streams to be distributed to another room in the conference enteredby a user or selected by said computer system; based on said conferencevariable data, said computer system generating a room list comprisingevery included conference room and having a highest room at a top ofsaid room list and having a lowest room at a bottom of said room list,and said computer system determining a number of rooms in the conferenceand a room type for every included conference room; based on saidconference variable data and said room list, said computer systemgenerating interconnection data for each included conference room, saidinterconnection data comprising instructions to interconnectteleconference equipment in each included conference room toteleconference equipment in every other included conference room; foreach included conference room, said interconnection data being generatedaccording to configuration information in a most preferable, feasibleset of configuration data among applicable conference configurationdata; for each included conference room, said most preferable, feasibleset of configuration data being selected by said computer systemperforming a recursive stream distribution process in which saidcomputer system steps through said room list from said highest room tosaid lowest room and, for each room, steps through any applicable setsof said configuration data in successive order from most preferable toleast preferable, according to said relative preference, until afeasible set of configuration data is found which permitsinterconnection to every other included conference room according to theinformation in the feasible set of configuration data, and if noapplicable set of configuration data is feasible for a current roombeing processed, then the distribution process returns to a next higherroom in said room list and selects an applicable set of configurationdata having a lesser relative preference than a previous set ofconfiguration data selected for said next higher room, and then thedistribution process again attempts to select a most preferable,applicable set of configuration data for said current room; andtransmitting said interconnection data to teleconference equipment ineach included conference room to initiate said conference.
 2. A methodof conducting an audio-visual conference among a number of conferencerooms, as in claim 1, wherein for at least room type, said configurationfactors includes a plurality of unique room modes.
 3. A method ofconducting an audio-visual conference among a number of conferencerooms, as in claim 1, wherein: said conference configuration dataincludes physical position indicators for each connected participantcamera to be connected to another included conference room, and includesindicators as to which other included conference room each connectedparticipant camera is connected, based on a relative position in saidroom list; and said configuration data indicates physical positionindicators for each connected participant display to be connected toanother included conference room, and includes indicators as to whichother included conference room each connected participant display isconnected, based on said relative position in said room list.
 4. Amethod of conducting an audio-visual conference among a number ofconference rooms, as in claim 1, wherein: after receipt of saididentities of said included conference rooms, said computer systemdetermines an optimal number of streams which may be commonly exchangedbetween all included conference rooms; and said computer system setssaid desired maximum number of streams equal to said optimal number ofstreams.
 5. A method of conducting an audio-visual conference among anumber of conference rooms, as in claim 1, wherein: after receipt ofsaid identities of said included conference rooms, said computer systemdetermines an absolute maximum number of streams which may be exchangedbetween two of said included conference rooms; and during receipt ofsaid conference variable data, said computer system presents said userwith means to select said desired maximum number of streams from onestream to said absolute maximum number of streams.
 6. A method ofconducting an audio-visual conference among a number of conferencerooms, as in claim 5, wherein: said computer system determines anoptimal number of streams which may be commonly exchanged between allincluded conference rooms; and in the absence of a selection of adesired maximum number of streams by said user, said computer systemsets said desired maximum number of streams equal to said optimal numberof streams.
 7. A method of conducting an audio-visual conference among anumber of conference rooms, as in claim 1, further comprising: saidcomputer system configuring teleconference equipment in a first room ofsaid included conference rooms to send first and second video streams toa second room of said included conference rooms, by configuring firstand second participant cameras in said first room to capture first andsecond groups of participants, respectively, where said first and secondgroups of participants are seated at first and second sets of seatinglocations, respectively; said computer system configuring teleconferenceequipment in said second room to display said first and second videostreams on first and second participant displays, respectively, in saidsecond room; first and second sets of participant microphones areassociated with said first and second sets of seating locations,respectively, in said first room; said computer system configuringteleconference equipment in said first and second rooms to reproduce afirst audio signal on a first participant speaker associated with saidfirst participant display in said second room, where said first audiosignal consists essentially of audio signals generated by said first setof participant microphones; and said computer system configuringteleconference equipment in said first and second rooms to reproduce asecond audio signal on a second participant speaker associated with saidsecond participant display in said second room, where said second audiosignal consists essentially of audio signals generated by said secondset of participant microphones.